The present invention relates to systems for providing information to the pilot of an aircraft and, in particular, it concerns a system for providing selected information to a pilot based on his gaze-direction without use of a visual display. In one application, the invention specifically addresses the control interface between a pilot and a weapon system through which the pilot designates and verifies tracking of a target by the weapon system.
The extremely high speed of modern air-to-air combat stretches the capabilities of a human pilot to their limits. Faced with complex aircraft instrumentation and high-tech weapon systems, a pilot is required to achieve split-second reaction times as supersonic aircraft pass each other at relative speeds up to thousands of miles per hour. Various high performance target-seeking air-to-air missiles have been developed to operate under these conditions. Nevertheless, the process of cueing such missiles and verifying that they are locked-on to the correct target before firing may be extremely difficult for the pilot, especially while simultaneously flying an aircraft under conditions of constantly varying orientation, extreme inertial forces and high stress.
To facilitate rapid designation of targets, a head-up display is typically used to indicate the current cueing direction. A display symbol representing the direction of regard of the missile seeker is brought into superposition with a directly viewed target and the seeker is then allowed to track the target. If the pilot sees that the display symbol is following the viewed target, he knows that the tracking is proceeding properly and can proceed to fire the missile.
Many state-of-the-art systems employ a helmet-mounted head-up display. In this case, the seeker typically follows an optical axis of the display which moves together with the helmet, the helmet position being monitored either by a magnetic or an optical system. Cueing is achieved by the pilot turning his head, and hence the helmet, to bring the optical axis into alignment with the target. Examples of such systems are commercially available, amongst others, from Elbit Ltd. (Israel) and Comulus (South Africa).
Despite the major technological advances which have been made in the implementation of helmet-mounted displays and cueing systems, such systems still suffer from a large number of disadvantages, as will now be detailed.
Firstly, the components mounted in the helmet add greatly to the weight of the helmet. This weight becomes multiplied numerous times under high-acceleration conditions, becoming a major source of fatigue and stress for the pilot.
Secondly, these systems generally require alignment of the optical axis of the helmet with the target to be designated. This limits operation of the system to the angular range of helmet motion which the pilot can achieve. This is typically smaller than the actual field of view both of the pilot and of the seeker of the air-to-air missiles, thereby limiting performance unnecessarily. Furthermore, shifting of the entire head together with the heavy helmet to the required angle under high acceleration conditions may require great effort, and may cause significant delay in the cueing procedure.
Thirdly, the helmet-mounted display typically requires very substantial connections between the helmet and other devices within the aircraft. These connections generally include a significant power supply and electrical and/or optical fibers for carrying projected information for the display. Such connections pose a significant safety hazard for the pilot, particularly with respect to emergency ejection where a special guillotine is required to sever the connections in case of emergency. The supply of a high voltage power line to within the helmet is also viewed as a particular safety hazard.
Finally, the integration of a head mounted display and cueing system into the aircraft systems is a highly expensive project, requiring adaptation of numerous subsystems, with all the complications of safety and reliability evaluation procedures and the like which this entails.
In addition to the specific issue of cueing and verifying correct tracking of weapon systems, modern aircraft include multiple information systems which in many cases generate information relating to objects or locations visible to the pilot. Such systems typically include radar and navigation systems of various types, as well as data systems. In many cases, DataLink (DL) systems are provided which can offer a wide variety of information, such as identifying other aircraft as friendly or hostile, identifying the type of aircraft and even provide information regarding the armament of the aircraft. Navigation related information typically includes the identity of various visible landmarks such mountains or cities. Commercially available examples of such systems in the U.S. include the systems known by the names xe2x80x9cLink4xe2x80x9d and Link16xe2x80x9d. In many cases it would be highly advantageous to provide this information on a head-up display so that it would be visually linked in an intuitive way to the pilot""s field of view. This however can only be achieved over a useful field of view by employing a helmet-mounted display with all of the aforementioned disadvantages.
Turning now to the field of eye-motion tracking, various techniques have been developed for identifying the gaze direction of the human eye. Examples of a number of commercially available systems for tracking eye movements may be obtained from ASL Applied Science Laboratories (Bedford, Mass., USA).
U.S. Pat. No. 5,583,795 to Smyth proposes a helmet-mounted apparatus for measuring eye gaze while providing a helmet-mounted display. Brief reference is made to the possibility of using the apparatus for xe2x80x9cdesignating targetsxe2x80x9d and xe2x80x9cweapon system pointingxe2x80x9d. Such a system, however, would still suffer from most of the aforementioned shortcomings associated with helmet-mounted display systems.
There is therefore a need for a gaze-actuated information system which would facilitate rapid and reliable cueing and tracking verification of air-to-air missiles without the pilot having to turn his entire head and without requiring substantial additional connections or expensive modification of aircraft systems. It would also be highly advantageous to provide a method for providing information, including confirming that a weapon system is locked-on to a visible target, without requiring use of a visual display.
The present invention is a gaze-actuated information system and method which provides information associated with various gaze directions within a field of view. Amongst other applications, the system and method may be used for confirming that a weapon system is locked-on to a visible target without use of a visual display. This allows the helmet-mounted parts of the system to be implemented as lightweight components, thereby rendering the helmet much lighter and easier to use than systems with helmet-mounted displays.
According to the teachings of the present invention there is provided, a method for providing a pilot with information associated with at least one region of a field of view visible to the pilot from within a cockpit without requiring a visual display, the method comprising the steps of: (a) determining an eye gaze direction relative to a given frame of reference for at least one eye of the pilot; (b) determining a reference direction relative to the given frame of reference; (c) comparing the eye gaze direction with the reference direction; and (d) if the eye gaze direction and the reference direction are equal to within a given degree of accuracy, generating audio output audible to the pilot and indicative of information associated with the reference direction.
According to a further feature of the present invention, the reference direction corresponds to a direction from a weapon system to a target to which the weapon system is locked-on, such that the audio output provides confirmation that the weapon system is locked-on to a target at which the pilot is currently gazing.
According to a further feature of the present invention, the reference direction corresponds to a direction from the cockpit to a friendly aircraft, such that the audio output provides an indication that an aircraft at which the pilot is currently gazing is friendly.
According to a further feature of the present invention, the reference direction corresponds to a direction from the cockpit to a hostile aircraft, such that the audio output provides an indication that an aircraft at which the pilot is currently gazing is hostile.
According to a further feature of the present invention, the reference direction corresponds to a direction from the cockpit to a landmark, such that the audio output provides information relating to the landmark at which the pilot is currently gazing.
According to a further feature of the present invention, the given degree of accuracy corresponds to a maximum allowed angular discrepancy between the eye gaze direction and the reference direction, the maximum allowed discrepancy having a value of less than 5xc2x0, and preferably less than 2xc2x0.
According to a further feature of the present invention, the determining an eye gaze direction includes: (a) employing a helmet-mounted system to derive direction information related to a relative eye gaze direction for at least one eye of the pilot relative to a helmet worn by the pilot; (b) transmitting the direction information via a cordless communications link to a receiver unit; (c) deriving position information related to a position of the helmet within a cockpit; and (d) processing the direction information and the position information to derive the eye gaze direction relative to a frame of reference associate with the cockpit.
According to a further feature of the present invention, the helmet-mounted system and a helmet-mounted portion of the cordless communications link are implemented using low-power electrical components powered exclusively by at least one helmet-mounted battery.
There is also provided according to the teachings of the present invention, a gaze-actuated information system for providing a pilot with information associated with at least one region of a field of view visible to the pilot from within a cockpit without requiring a visual display, the system comprising: (a) a gaze-direction determining system deployed within the cockpit and configured to determine a current gaze direction of the pilot relative to the cockpit; (b) a direction correlation system associated with the gaze-direction determining system and configured to compare the current gaze direction with at least one reference direction and to generate a correlation signal when the current gaze direction is equal to the reference direction within a predefined margin of error; and (c) an audio output system associated with the direction correlation system and configured to be responsive to the correlation signal to generate audio output audible to the pilot and indicative of information related to the reference direction.
According to a further feature of the present invention, there is also provided a weapon system including a seeker operative to track a target, the weapon system generating a current target direction corresponding to the direction from the seeker to the target being tracked, the direction correlation system being associated with the weapon system and configured to employ the current target direction as one of the reference directions such that, when the pilot looks towards the target, the audio output system generates audio output indicative that the currently viewed target is being tracked.
According to a further feature of the present invention, the gaze-direction determining system includes: (a) a helmet-mounted system configured to derive relative direction information related to a relative eye gaze direction for at least one eye of the pilot relative to a helmet worn by the pilot; and (b) a helmet positioning system configured to derive position information related to a position of the helmet within the cockpit.
According to a further feature of the present invention, the gaze-direction determining system further includes a transmitter deployed for transmitting a wireless signal containing information from the helmet-mounted system.
According to a further feature of the present invention, the helmet-mounted system and the transmitter are implemented using low-power electrical components powered exclusively by at least one helmet-mounted battery.
There is also provided according to the teachings of the present invention, a method for providing to a pilot confirmation that a weapon system is locked-on to a visible target without use of a visual display, the method comprising the steps of: (a) determining an eye gaze direction relative to a given frame of reference for at least one eye of the pilot; (b) determining a target direction representing the direction relative to the given frame of reference from the weapon system to the target to which the weapon system is locked-on; (c) comparing the eye gaze direction with the target direction; and (d) if the eye gaze direction and the target direction are equal to within a given degree of accuracy, generating a predefined audible signal to confirm that the weapon system is locked-on to a target at which the pilot is currently gazing.
According to a further feature of the present invention, the given degree of accuracy corresponds to a maximum allowed angular discrepancy between the eye gaze direction and the target direction, the maximum allowed discrepancy having a value of less than 5xc2x0, and preferably less than 2xc2x0.
According to a further feature of the present invention, the determining an eye gaze direction includes: (a) employing a helmet-mounted system to derive direction information related to a relative eye gaze direction for at least one eye of the pilot relative to a helmet worn by the pilot; (b) transmitting the direction information via a cordless communications link to a receiver unit; (c) deriving position information related to a position of the helmet within a cockpit; and (d) processing the direction information and the position information to derive the eye gaze direction relative to a frame of reference associate with the cockpit.
According to a further feature of the present invention, the helmet-mounted system and a helmet-mounted portion of the cordless communications link are implemented using low-power electrical components powered exclusively by at least one helmet-mounted battery.